1. Field
The present invention is directed to a robotic lawn mower and a training method thereof. The robotic lawn mower can detect and identify obstacles and can autonomously avoid the detected obstacles while operating. In particular, the robotic lawn mower is able to be trained, in order to learn new obstacles for improving its visual obstacle detection. A profound user-mower interaction enables a user to customize the robotic lawn mower.
2. Description of the Related Art
Autonomous or robotic lawn mowers are an increasing market. Such mowers autonomously mow the lawn e.g. in a random fashion cutting only small pieces of grass in every run (“mulching”). The small pieces fall into the sod, thereby fertilizing the lawn. This principle is called mulching. Usually a mowing area is delimited by an electric border wire, which emits a weak electromagnetic field. This electromagnetic field is detected and used by the mower to stay within the allowed mowing area or to find a base station for recharging. To avoid obstacles (either static or dynamic), which are not indicated by a border wire, autonomous mowers typically use bump and sonar sensors.
Unfortunately, bump and sonar sensors work reliably only for large obstacles. Flat obstacles like smart phones or hoses are not recognized, and may be damaged when the mower drives over them. This drawback can potentially be removed by attaching a camera on the mower and recognizing obstacles by means of image processing. One typical solution in this respect is to provide a classifier module of the mower with samples of grass and obstacles so that the classifier module is able to distinguish between drivable grass and non-drivable obstacles.
The problem of such classifying approaches is that it is difficult to foresee all potential kinds of obstacles that may be encountered during the operation of the mower. Every kind of obstacle that has not been provided to the classifier module beforehand may be later miss-evaluated during operation.
EP 2 286 653 describes an autonomous lawn mower that combines camera grass recognition with the output of a grass sensor. The grass sensor is used to bootstrap and update a classifier that classifies pixels either as grass or non-grass based on their pixel values (e.g. color). Furthermore, a computer implementation for controlling a mower with a camera is described. The classifier of the mower cannot in any way be influenced by a user. The mower can also not recognize different classes of obstacles, and consequently cannot trigger different behaviors of the mower regarding different obstacles.
WO 2010/077198 describes an autonomous robotic lawn mower and a method for establishing a wireless communication link between the lawn mower and a user. A border-wire-based robotic mower is disclosed and is able to communicate with a cellular network structure. The mower can inform the user when the mower detects a “situation”. The user can send back an acknowledgement signal or a message. However, no information for clarification of an uncertain detection result or for improving a classifier installed in the mower software is mentioned. The user is not able to improve the classifier, and also different obstacles cannot trigger different behaviors of the mower.
U.S. Pat. No. 6,611,738 B2 describes a general purpose mobile device, whose navigation is based on GPS. A movement teach-in is disclosed based on the GPS data, where a teach-in is a well-known technique in the field of robotics for teaching a robot to mimic a certain sequence of motor commands or motor positions by moving a robot manually or by means of a remote control. The user can give additional information about the working area, like the position of a newly planted tree. However, training and improving of the mower's visual obstacle detection by the user is not possible.
US 2010/0324731 A1 describes a method for establishing a desired area of confinement for an autonomous robot, and an autonomous robot implementing a control system for executing the same. An already known method of movement teach-in for acquiring a user-defined perimeter is described. Further, an interaction between a user and the mower via a smart phone (or similar devices) is disclosed. First, a map generated by means of perimeter information can be displayed on the smart phone. Second, the mower may be remotely controlled via the smart phone. Third, the base station may act as a communication gateway between smart phone and mower. However, interactions concerning the visual recognition of the mower are not anticipated. In particular, the smart device cannot display the visual input of the mower, so that the user cannot give any input for updating a classifier.
“Identification & Segmentation of Lawn Grass Based on Color and Visual Texture Classifiers”, Master Thesis, Alexander Schepelmann, describes a grass recognition system for an autonomous lawn mower (CWRU cutter, Case Western Reserve University). The grass detection is based on color and texture cues. The approach employs classifiers, which are trained on some sample images. This approach includes a typical learning phase for training a classifier. However, user feedback to adapt the classifier or any attempts to identify different classes of obstacles is not envisaged.
WO 2011/002512 describes visual grass recognition based on image statistics. One idea is that the statistics in a current camera image are compared to the statistics of grass that were recorded in advance. This is a typical example of a predefined detection, which cannot solve the problems addressed by the present invention, because each garden has its own individual properties.